CC-42186: Sanitize DLQ WriteException/WriteConcernException (drop document BSON details) - #114
Draft
Sharan Rudresh (sharanrudresh) wants to merge 1 commit into
Draft
Conversation
…ception PR #110 sanitized the ERROR log statements but left the DLQ exception classes untouched. WriteException / WriteConcernException still embedded the failing document's BSON (error.getDetails().toJson()) in getMessage(), which surfaces: - at DEBUG via StartedMongoSinkTask.log() (LOGGER.debug(..., e)), and - in DLQ error headers via AnalyzedBatchFailedWithBulkWriteException.report(). Drop the `details` field from both message formats and bump the documented message-format version 1 -> 2 (the `v` marker exists precisely to signal such format changes). code / codeName / message are retained. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sanitizes the two DLQ exception classes so that the failing document's BSON is no longer leaked:
WriteExceptionWriteConcernExceptionBoth previously embedded
error.getDetails().toJson()(the failing document's BSON) directly ingetMessage(). This change drops thedetailsfield from both message formats and bumps the documented message-format version 1 → 2. Thecode/codeName/messagefields are retained.Why this is needed (the gap #110 left)
#110 deliberately scoped its fix to the ERROR log statements and explicitly left the DLQ exception classes untouched. As a result the document BSON still escaped via two paths that #110 did not close:
StartedMongoSinkTask.log()doesLOGGER.debug("…(full detail)", records.size(), e), whereeis theWriteException; itsgetMessage()(withdetails=<BSON>) prints in full at DEBUG.AnalyzedBatchFailedWithBulkWriteException.report()→errorReporter.report(record, writeException), so the exception message (with BSON) lands in the DLQ record's error headers.Message-format version bump (reviewer note)
Removing the
detailsfield is a breaking change to the documented DLQ message contract, soMESSAGE_FORMAT_VERSIONis bumped1 → 2and the javadoc updated — thev=marker exists precisely to signal this. Any downstream consumer parsing thev=1format (with a trailingdetails=…) should key off the version.Scope
detailsonly. The driver'smessagestring is retained. Note: for duplicate-key errors the driver'smessagecan itself echo the conflicting key value — sanitizing that further was considered and intentionally left out of scope for this PR.StartedMongoSinkTaskTestreferences these classes by type (new Report(idx, WriteException.class)), not by message string, so no test changes were required. (Build/tests not run locally — relying on CI.)Related
This is the follow-up flagged on the backport PR #113 (which cherry-picked #110 to
v1.16.x). This PR targetsv2.0.x; a matchingv1.16.xchange can follow if desired.JIRA
StartedMongoSinkTasklogs full Mongo write-error documents at ERROR (fix rec: dropdetails.toJson()fromWriteException.getMessage())AnalyzedBatchFailedWithBulkWriteExceptiondispatches document BSON to the ERROR log (fix rec: strip BSON details fromWriteException.getMessage())Originating work: #110.